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ABSTRACT 

Two  distinct  subjects  are  discussed  In  the  report, 

1.  The  notion  of  a  mathematical  flow  diagram  for  describing 

a  computational  procedure.  Introduced  by  von  Neiojnann  and  cc-workers 
In  19^6,  Is  discussed.  The  rules  for  constructing  and  Interpreting 
such  a  diagram  are  given. 

2.  Various  methods  for  the  estimation  of  significance  In 
numerical  calculations  are  summarized,  with  comments. 


This  material  was  prepared  in  collaboration  with  N,  C  Metropolis, 
for  possible  Inclusion  as  an  appendix  to  a  proposed  book  on  Monte 
Carlo  methods. 
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1.  FLOW  DIAGRAMS 
2.  THE  ESTIMATION  OF  SIGNIFICANCE 

1.  Introduction. 

About  15  years  ago,  von  Neumann  and  his  co-workers  felt 
the  need  of  a  more  precise  way  of  formulating  large  computational 
problems  than  via  the  existing  mathematical  terminology,  and  they 
devised  a  way  of  representing  each  problem  by  a  "flow  diagram." 
Since  then,  the  original  objective  appears  to  have  been  lost 
sight  of,  to  some  extent,  partly   because  von  Neumann  never 
recorded  his  Ideas  on  the  subject  with  the  completeness  and 
clarity  that  characterized  most  of  his  work. 

We  feel  that  it  will  be  useful  to  set  forth  once  more  the 
original  ideas,  following  quite  closely  the  reports  of  Burks, 
Goldstine  and  von  Neumann  (1946,  194?),  but  with  minor  modifi- 
cations and  amplifications  aimed  at  greater  clarity,  in  line, 
we  hope,  with  von  Neumann's  unwritten  as  well  as  his  written 
comments  on  the  subject. 

The  objective  was  to  furnish  a  precise  formulation  of  an 

2 
abstract  computational  procedure,   not  a  mnemonic  guide  to  the 

operation  of  a  computer.   Conceivably,  the  technique  could  be 

useful  in  other  branches  of  mathematics,  where  a  complicated 

2 

During  the  formulation,  it  is  often  wise  to  keep  in  mind  the 

^characteristics  (memory  size,  etc.)  of  the  computer  to  be 

used,  but  it  was  von  Neumann's  feeling  that  a  complete  abstract 

formulation  should  be  made.  Independent  of  detailed  planning 

for  the  computer,  human  or  electronic. 
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pattern  of  operations  or  procedures  has  to  be  definedo 

The  need  arises  because  most  mathematics  is  primarily 
concerned  (at  least  today)  with  statements  of  fact,  rather 
than  the  performance,  or  even  formulation,  of  specific 
calculations,  numerical  or  analytic o^  Mathematics  has 
probably  never  been  concerned  with  calculations  of  the  size 
and  complexity  of  many  that  are  nowadays  entrusted  to  computers. 
One  should  not  expect  that  standard  notations  and  modes  of 
expression  would  be  suitable  or  even  adequate  for  that  purpose, 
although  they  can  often  be  made  so  by  supplementing  the  formulas 
with  verbal  comments  such  as,  "for  each  J  in  succession 
(J  =  0,1,0 «o),  one  solves  equation  (I7)  for  x  and  substitutes 
the  result  intOooo"|  in  very  large  problems,  this  is  ctuiibersoiae . 

We  shall  limit  discussion  to  flow  diagrams  satisfying  the 
following  criteria?   (l)  the  intention  is  to  give  a  cossplete  and 
precise  description  of  all  calculational  stepsj  (2)  references 
to  a  particular  computer,  if  any,  play  a  secondary  role.   The 
first  criterion  demands,  obviously,  that  all  notational 
conventions  used  be  exactly  defined,  and  we  shall  define  a  set 
of  conventions  very  similar  to  the  ones  used  originally  by  Burks p 
Golds tine,  and  von  Neumann. 

A  flow  diagram  is  no  guarantee  that  a  perfect  computer  code 

An  important  exception  would  seem  to  be  the  tendency,  among 
many  mathematicians,  to  be  unhappy  about  an  existence  proof 
that  is  not  constructive  -  i.e.,  does  not  indicate,  in 
principle,  by  what  steps  one  can  construct  the  particular 
mathematical  object  whose  existence  is  being  proved. 


will  be  wrltteno   Neither  is  it  necessary:  some  people  can 
plan  and  code  a  problem  effectively  without  onej  however, 
many  people  find  it  convenient  to  formulate  a  problem  directly 
In  the  form  of  a  flow  diagram..   (For  this  purpose^  a  special 
pencil  is  used^  which  is  half  pencil  and  half  eraser.)   In 
any  case,  it  is  our  point  of  view  that  only  by  means  of  a 
proper  flow  diagram  cam  the  calculatlonal  procedure  be 
adequately  described  or  communicated  to  another  person  short 
of  giving  a  complete  computer  code  or  the  equivalent. 

Diagrams  of  another  kind^,  called  flow  charts  are  often  used 
in  addition^  they  make  little  attempt  to  describe  the  contents 
of  the  operation  boxes,  except  by  words  such  as  "calculate 
stream  function"  or  "read  new  block  of  data  from  tape  5"*  but 
emphasize  mainly  the  overall  sequencing  and  various  circumstances 
having  to  do  with  the  operation  of  a  particular  computer.   They 
are  helpful  in  checking  out  and  running  a  problem,  but  should 
not  be  confused  with  flow  diagrams  in  the  original  sense. 

2.  Rules  for  Flow  Diagrams. 

The  routine  of  a  calculation  of  sufficient  magnitude  and 
complexity  to  warrant  the  use  of  a  modern  computer  usually  has 
a  complicated  set  of  interconnections  among  its  various  parts, 
most  of  which  are  used  many  times  over  in  varying  contexts. 
The  proper  sequencing  of  the  operations  in  the  computer  is 
effected  by  jump  instructions ^  conditional  and  unconditional, 
which  provide  the  interconnections  referred  to.   Each  elementary 
part  of  the  calculation  consists  of  a  prearranged  sequence  of 
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steps  without  repetitions p  gaps,  or  alternatives!  such  a  sequence 
can  be  specified  by  a  list  (for  example^  in  computer  code)  of 
the  steps,  written  one  over  the  other  in  a  vertical  coliinm,  or 
horizontally  one  after  the  other,  or  in  any  other  linear 
arrangement o   On  the  other  hand,  if  the  description  of  the 
complete  routine  is  written  in  a  linear  or  one=dimensional 
fashion,  or  list 5,  the  sequence  of  items  in  the  list  fails  to 
represent  the  sequence  of  steps  in  the  calculation  whenever 
a  Jump  instruction  occurs »   In  order  that  the  geometrical 
arrangement  of  the  items  should  give  a  topologically  true 
representation  of  the  calculation^  two  or  more  dlmejtiaions  are 
needed,  to  display  loops^  branching,  merging,  and  the  likeo 
Two  dimensions  will  always  suffice,  at  least  If  one  allows 
flow  lines  to  cross  each  other  without  interference,  as  In 
electrical  circuit  diagrams =   Hence,  a  two-dimensional  figure 
is  natural  for  a  description  of  the  sequencing  of  a  calculation » 

The  linear  portions  of  the  calculation  are  specified  in 
operation  boxes,  by  means  of  a  symbolism  to  be  described 
presently,  and  the  boxes  are  connected  by  flow  lines  (bearing 
arrowheads  to  indicate  their  direction) ,  according  to  a  pattern 
determined  by  the  structure  of  the  problem.   Branching  is 
achieved  by  means  of  condition  boxes ,  which  have  two  (or  more) 
exits  J  the  computer  follows  the  one  or  the  other  exit  flow  line 
according  to  whether  some  specified  condition  is  satisfied 

k. 

We  use  this  word  loosely,  and  trust  that  the  reader  will 

understand  what  is  meant o 
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(e.g.,  according  to  whether  some  quantity  It  has  Just 
calculated  is  positive  or  negative) .  'The  connections  are 
subject  to  a  few  rules  that  are  rather  self-evident:  an  operation 
box  has  Just  one  exit  (except  for  a  box  containing  stop  as  its 
last  item,  which  has  no  exit)j  flow  lines  may  merge,  but  not 
branch  out,  between  boxes j  if  two  flow  lines  enter  a  box,  the 
effect  is  the  same  as  if  they  had  merged  before  entering;  the 
items  within  a  box  are  Invariably  taken  in  the  order  in  which 
they  were  written  (i.e.  from  top  to  bottom  and,  within  each 
row  of  writing  containing  two  or  more  items,  from  left  to  right 
within  the  row) ,  regardless  of  where  the  flow  lines  enter  and 
leave  the  box.   In  the  case  of^ a. special  device  called  a  remote 
connection,  to  be  described  later,  computer  control  Jumps  from 
one  specified  point  in  the  diagram  to  another  without  following 
any  visible  path;  this  is  admittedly  contrary  to  the  basic  notion 
of  a  flow  diagram  but  is  necessary,  if  the  diagraan  will  not  fit 
onto  a  single  sheet  of  paper ;,  and  is  sometimes  convenient  for 
other  reasons. 

To  illustrate  the  need  for  careful  definition,  let  us 
suppose  that,  in  describing  a  problem,  one  has  written; 

(1)  g  =  ax^  +  bx  +  ex  +  d. 

This  is  presumably  intended  to  mean  that  the  numerical  values 
assigned  to  the  variables   a,  b,  c,  d,   and  x,   are  to  be 
combined  in  the  manner  indicated  on  the  right  of  the  equation, 
and  the  resulting  numerical  value  is  to  be  assigned  to  the 
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variable  go   However,  In  a  typical  computer  calculation,  the 
number  of  numbers  (numerical  values)  occurring  vastly  exceeds 
the  number  of  variables,^   It  Is  therefore  highly  likely  that 
all  six  of  the  variables  a^  b,  c,,  d,  x,   and  g.,   have  had 
numerical  values  assigned  to  them  at  the  time  equation  (1)  Is 
encountered^  generally  these  values  will  not  satisfy  equation 
(l),  and  one  of  thera  has  to  be  changedo   We  have  taken  It  for 
granted.  In  the  present  case^,  (from  our  knowledge  of  the 
psychology  of  mathematicians)  that  It  Is  g,   rather  than 
a^  b,  c>   d,   or  x^,   that  has  to  change^   But,  without  some 
clear  convention,  one  Is  not  Justified  in  placing  any  particular 
Interpretation  on  (1)  as  an  operational  instruction.,   Even  our 
knowledge  of  psychology  would  be  of  no  use  If  the  equation  were 

(2)  f (x)  =  g(y) 

or 

(5)  a  -  b 

(Such  equations  have  been  seen  in  actual  problem  descriptions!) 

One  must  either  redefine  the  symbol   s,   for  use  in  flow 
diagrams,  or  introduce  a  new  oneo   Because  of  the  usual  static 
reflexive  meaning  of  the  sign  =,   we  prefer   to  introduce  a 

■^  ^  a   variable  Is  meant  a  symbol  (usually  literal)  to  which 
different  values  can  be  assigned  at  different  times »   Owing 
to  the  small  number  of  available  symbols,  most  of  them  generally 
have  to  be  used  many  times  over. 

Many  people  continue  to  use  the  sign  =  with  a  definite  under- 
standing as  to  which  side  of  the  "equation"  is  to  be  changed » 
This  is  perfectly  sound,  but  we  want  to  emphasize  here  that 
we  are  dealing  with  an  instruction,  not  a  statement  of  facto 
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new  symbol  — >  (pronounced  "for"),  whose  meaning  is  governed 
by  a  set  of  rules,  as  follows:  (the  symbol  =  will  reappear 
later  in  a  different  connection). 

Rule  1.  When  the  symbol  — >  stands  between  two  variables, 
it  means  that  henceforth,  until  further  notice,  the  variable 
on  its  right  is  to  have  assigned  to  it  the  numerical  value 
obtained  from  the  variable  (or  combination  of  variables)  on 
its  left.   In  other  words,  the  variable  to  be  changed  always 
stands  on  the  right.   If  the  variable  on  the  right  appears  in 
the  combination  of  variables  on  the  left,  as  in  the  familiar 
case   "n  +  1  — >  n" ,   the  old  value  is  of  course  to  be  used 
there  for  computing  the  new  one;  otherwise,  all  variables 
appearing  on  the  left  continue  to  retain  their  previous  values. 

With  this  rule,  equation  (1)  would  be  rewritten  as 

(k)  ax^   +  bx  +  ex  +  d  ^'  g; 

and  equation  (3)  would  be  rewritten  either  as 


b, 

or 

b  — >  a, 

depending  on  which  is  Intended. 

(We  are  concerned  mainly  with  abstract  flow  diagrams. 
But,  if  one  has  a  particular  computer  in  mind  and  wjshes  to 
indicate  which  memory  location  a  quantity  is  to  be  stored  in, 
the  word  "to"  is  used,  not  an  arrow.   This  may  be  combined 

-  10  - 


with  a  step  of  the  calculation.  If  desiredi  for  example,,  the 
expression   "x  +  y  -^  z  to  51I"  means  that  the  sum  of 
X  +  y  is  to  be  stored  in  memory  location  5II  and  is  to  be 
referred  to  by  the  symbol  z  in  subsequent  parts  of  the  flow 
diagram,) 

The  variable  may  be  a  single  letter  or  a  compound  symbol, 
like  y^  or  p.^^  or  f(x)^   in  which  a  functional  relation 
is  implied?  we  then  call  It  a  compound  variable..   When  its 
arguments  (loeo   n^   or  J  and  k,   or  x)   have  had  values 
assigned  to  them^  it  behaves  just  like  a  simple  variable  until 
the  value  of  one  or  more  of  its  arguments  changes  i,  in  which 
case  the  following  rule  applies s 

Rule  2o  When  the  value  of  a  variable  is  modified^  this 
modification  is  understood  to  take  place  automatically  in  all 
compound  variables  that  depend  on  ito   For  example 5,  if,,  at 
some  stage  in  a  calculation,,   n  has  the  value  3^,   T   stands 
for  T^  and  T  1   stands  for  Tuf,      if  a  substitution 
n  +  1  -->  n  is  then  made  so  that  n  assumes  the  value  Kg    the 
symbol  T   automatically  comes  to  stand  for  Tu      and  T  t 
for  Tp-j   it  is  unnecessary^  and  in  fact  Incorrect g  to  write 
"T  ,  — >  T  "   after  writing  "n  +  1  -^  n" ,,   unless  one  intends 
not  only  to  alter  n  but  also  to  alter  Tu      so  as  to  make  it 
equal  to  T^- 

When  a  compound  variable  stands  on  the  right  of  the  symbol 
— >  ,   a  further  rule  is  needed  to  make  the  meaning  unique.. 

Rule  5o   When  a  compound  variable  representing  a  function 
of  one  or  more  arguments  stands  on  the  right  of  the  symbol  — >  j, 
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It  Is  the  functional  value  that  Is  to  be  changed,  not  an 
argument. 

For  example,  at  a  time  when  J  has  the  value  1?  and  N 
the  value  6,  the  expression  fT  -^  n.  means  that  the  variable 
n,  ^  Is  to  be  assigned  the  value  ^6,  even  though  it  might  be 
possible,  by  changing  the  value  of   J,   to  find  some  other 
variable  n.   (say  n,^)   that  is  already  equal  to  36. 
Similarly,  at  a  time  when  x  has  the  value  1.7,   the  substitution 
0.25  -^   f (x)   means  that  henceforth  f (1-7)  =  0.25;  i-e.  it  means 
that  the  function  is  to  be  changed  (at  this  one  value  of  x) ,  not 
that  we  should  solve  the  equation  f (x)  =  0.25  to  find  a  new 
value  of  X,   using  the  old  function. 

Occasional  exceptions  are  allowed  in  the  case  of  functions 
whose  values  and  notation  are  permanently  established  by  rigid 
mathematical  convention,  so  that  the  argument  must  change. 

If  f (x)   in  the  last  example  is  the  function  x  ,   the  substi- 

2 
tutlon  0.25  — >  X   must  clearly  mean  that  the  value  of  x  is 

to  be  changed  from  I.7  to  0.5   (or  possibly  to  -0.50>  and 

does  not  imply  that  we  are  trying  to  change  the  meaning  of  the 

2 
function  denoted  by  x   at  the  argviment  x  =  I.7,   so  as  to 

make  "(1.7)^"  mean  "0.25"   rather  than  "2.89."   Similarly, 

the  expression  —5 —  -^   sin  0  would  be  taken  to  require  a 

a^+1 
modification  of  the  value  of  ©,   However,  it  is  best  to  avoid 

all  exceptions  to  Rule  3  where  there  can  be  possible  doubt  about 

the  intention. 

It  was  noted  above  that  the  items  in  a  box  are  always  to 

be  taken  in  the  order  written.   This  permits  one  step  in  a  box 

-  12  - 


to  modify  variables  appearing  In  later  steps  In  the  same  box, 
as  for  example  In 


■> 


We  have  been  Intentionally  vague  as  to  how  complicated  an 
expression  is  permitted  to  occur  on  the  left  of  the  symbol  — >  = 
We  do  not  wish  to  be  restricted  to  expression  that  can  be 
evaluated  by  an  unbroken  sequence  of  coding,  because  computers 
vary  greatly  as  to  what  can  be  accomplished  by  a  single 
instruction  or  elementary  operation.   Some  computers  have  built- 
ir  circuits  for  square  roots  and  exponentials j  whereas,  at  the 
other  extreme,  even  division  requires  a  coded  subroutine  in  some. 
We  are  aiming  at  an  abstract  description  of  a  calculations 
independent  of  computers,  and  the  only  requirement  in  the  present 
connection  is  that,  if  there  is  any  doubt  as  to  what  calculational 
steps  are  needed  to  evaluate  an  expression  used,  the  diagram 
should  be  expanded  to  show  the  steps  in  more  detail. 

If  we  regard  an  instantaneous  state  of  a  calculation  as 
being  determined,  for  purposes  of  abstract  description,  when  some 
or  all  of  the  variables  have  specified  numerical  values  assigned 
to  them,  it  Is  clear  that  the  calculation  progresses  from  one 
state  to  another  only  during  a  step  described  by  an  expression 
containing  the  sign  -^   .   We  call  such  a  step  a  (generalized) 
substitution,  because,  generally,  one  numerical  value  is 
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substituted  for  another  as  the  meaning  of  some  variable. 

It  may  be  noted  parenthetically  that  the  Instantaneous 

state  of  a  given  computer  may  have  to  change  many  times  during 

such  a  substitution,  as  Intermediate  quantities  on  the  left  of 

the  sign  — >  are  computed  and  then  used  to  obtain  the  value 

of  the  full  expression.   On  the  other  hand,  the  state  of  the 

machine  may  not  change  at  all,  as  In  the  case  of  a  substitution 

n  +  1  — >■  n  If  the  value  of  n  Itself  does  not  have  to  be 

computed  or  stored;  In  such  a  case  numerical  quantities  associated 

with  compound  variables  like  x   have  their  nsunes  changed  while 

they  are  sitting  quietly  In  the  storage  of  the  computer.   E.g.  the 

number  that  was  formerly  denoted  by  x   will  have  to  be  called 

n 

x  ,   the  next  time  It  Is  referred  to.   We  wish  to  emphasize 
n-1  ^ 

again  that  the  flow  diagram  is  Intended  to  give  an  abstract 
description  of  a  calculation,  not  a  coding  procedure. 

In  contrast  to  substitution,  by  means  of  which  a  calculation 
progresses,  we  have  statements,  containing  relation  signs  like  =  , 
<>    >f   <>   >>    =•  ^'  etc.   which  are  simply  either  true  or  false  but 
do  not  indicate  that  any  variable  is  to  have  its  value  changed. 
Statements  are  of  two  types  --  absolute  and  conditional. 

Rule  4.  An  absolute  statement  has  no  function  or  effect 
other  than  to  remind  the  reader  of  something  he  may  have  forgotten 
or  to  point  out  the  physical  or  other  interpretation  of  a  particu- 
lar part  of  a  calculation.   (Therefore  it  need  not  be  complete  or 
precise.)   It  should  be  put  in  parentheses  and  preferably  beside 
a  box  or  a  flow  line  rather  than  inside  a  box.   For  example,  if 
an  induction  on  n  =  1,  2,...,  N  is  terminated  not  by  testing 
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whether  n  =  N  but  by  some  other  test  that  is  passed  only 
when  n  =  N^  one  may  wish  to  write  the  statement  "(n  =  N) " 
near  the  exit  line  of  the  subroutine j,  viZo,  ^^  ^  ^f >  , 


or  one  may  wish  to  include  a  descriptive  phrase  like  "(fission)" 
or  "(inelastic  scattering)",  to  tell  the  physical  significance 
of  a  test  or  of  the  calculation  in  a  particular  box,, 

Rule  5°  A  conditional  statement  is  a  statement  occuring 
in  a  box  having  two  exits ^  labelled  "yes"  and  "no"  (or  equivalently) ^ 
it  being  understood  that  the  calculation  takes  the  first  exit  if  and 
only  if  the  statement  is  trueo   Some  people  put  a  question  mark 
after  a  conditional  statements  and  some  do  not.   A  conditional 
statement  may  contain  just  one  variable,  as  "  x  >  0"  or  several, 
as   "k  =  i(mod  n)"o   For  simplicity  and  economy  of  spacej,  one 
often  combines  a  condition  box  with  the  operation  box  that  precedes 
it,  as  in  the  example  on  page  17°   In  this  case  the  conditional 
statement  must  be  the  last  item  in  the  box. 

Multiple  branching  can  be  achieved  by  testing  one  condition 

after  another.   However >,  one  case  is  sufficiently  common  to 

deserve  a  special  notationi  in  it,   "s"  is  used  as  a  "compare" 

symbol,  and  the  exit  lines  are  labelled  with  relation  symbols, 

as  follows. 

> 


< 


a  %   b 


> 


-> 


15  - 


This  device  can  be  used  whenever  the  conditional  statements 
are  mutually  exclusive  and  each  can  be  expressed  by  a  relation 
symbol  standing  between  two  specified  variables. 

Occasionally,  a  computer  is  allowed  to  calculate  for  a  very 
brief  period  with  undefined  quantities  (e.g.  with  whatever  numbers 
happen  to  be  left  over  in  the  computer  from  previous  operations) 
if  one  can  be  sure  that  the  quantities  so  calculated  will  either 
be  ignored  or  multiplied  by  zero.   For  example,  the  first  step 
of  an  iteration  may  require  only  part  of  a  subroutine  used  in 
later  steps,  but  it  may  simplify  the  coding  (at  the  expense  of 
a  slight  increase  in  computing  time)  to  allow  the  computer  to 
go  through  the  entire  subroutine.   Similar  occurrences  may  be 
allowed  also  in  an  abstract  flow  diagram;  but  otherwise  every 
symbol  occurring  In  an  expression  on  the  left  of  a  sign  -^ 
should  have  had  some  numerical  value  assigned  to  it  previously: 
i.e.,  it  should  denote  either  the  result  of  some  previous  step 
or  an  input  datum. 

Rule  6.  Variables  denoting  the  input  data  of  a  problem 

should  be  written  in  a  list,  in  a  special  kind  of  box  resembling 

a  huge  pair  of  parentheses  and  connected  to  the  flow  line  by  a 

dashed  line,  as  in  the  illustration:         /  \ 

'  a, 
b, 
N, 


START 


'    ^N 


etc. 


The  input  data  are  the  free  variables  of  a  calculation.   If 
storage  locations  (either  absolute  or  symbolic)  for  a  particular 
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program  are  also  Indicated,  the  list  becomes  a  storage  chart,  viz, 

\ 


START 


V 


200  %    a 

201  i    b 

202  I    N 
500  +  n  s  X 


n 


(n  =  0,  1,  '-,    N) 


Similar  lists  or  storage  charts  may  appear  elsewhere  in  the 
diagram « 

Further  conventions  may  be  used,  more  or  less  at  will, 
provided  the  meaning  is  clear o   For  example,  the  expression 


"0  ->  x 


n 


:n  -  0,  1. 


N)" 


occurring  in  a  box,  does  not  conform  to  the  preceding  rules, 
but  is  an  obvious  abbreviation  for  the  steps g 


0  ->  n 


However,  great  care  must  be  exercised  to  notice  exactly  which 
variables  have  their  values  changed.   The  above  set  of  steps 
is  different  from  the  set 
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and  may  actually  lead  to  different  results  in  a  calculation, 

because  the  variable  n  generally  has  its  value  changed  in 

the  first  case  but  does  not  even  appear  in  the  second.   Similarly, 
2  .     2 


the  steps  x'"  ^.  y,  y*"  — >  z  will  generally  have  a  different  effect 

4 
from  the  single  step  x  — >  z   if  the  symbol  y  has  had  a  value 

assigned  to  it  previously  and  is  to  be  made  use  of  subsequently. 

It  is  for  this  reason  that  we  regard  a  generalized  substitution 

as  the  basic  step  of  an  abstract  description  of  a  calculation. 

Two  more  subjects  must  be  discussed  before  we  are  finished 
with  flow  diagrams?  they  are  remote  connections  and  a  special 
symbolism,  used  in  Monte  Carlo  work,  to  denote  the  act  of  sampling 
a  random  distribution. 

A  remote  connection  is  simply  the  replacement  of  a  long 
flow  line,  viz. , 


by  the  following  diagramatic  device. 


0 


-> 


which  has  the  same  meaning.   The  flow  Jumps  from  the  first 
circle,  called  the  exit  point,  to  the  second  circle,  called 
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the  entrance  pointy  or,  more  generally  to  that  particular 
entrance  point  bearing  the  same  symbol  as  the  exit  point  that 
it  came  from  (the  letter  "a"  above).. 

This  device  is  convenient  if  the  flow  diagram  occupies 
two  or  more  sheets  of  paper »   Furthermore,  it  can  be  made  into 
a  variable  remote  connection^  by  having  several  possible 
entrances,  e.g. , 


,*i 


->  b 


CO- 


V 


so  that  the  flow  can  Jump  to  any  one  of  the  entrances ^  depending 
on  circumstances  (this  is  simply  a  convenient  way  of  achieving 
multiple  branching) .   The  flow  diagram  itself  must  somehow 
specify  which  alternative  is  to  be  followed »   This  Is  done  by 
regarding  the  symbol  of  the  exit  point  as  a  variable  whose  value 
is  to  be  established  by  a  substitution  according  to  the  same 
rules  as  for  niimerical  variables,.   Thus,  the  substitution 


'  b. 


in  a  box  of  the  flow  diagram,  means  that  henceforth,  until 


further  notice,  the  variable  exit  point 
identified  with  the  entrance  point   f b^ i 


b    is  to  be 

Variable  connections 
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are  thus  always  prepared  in  advance  and  c£in  be  modified  at 
any  time.   A  few  obvious  rules  are 

Rule  7.  No  two  entrance  points  may  have  the  same  symbol. 

Rule  8.  A  variable  exit  may  not  have  the  saune  symbol  as 
any  entrance  (for  then  It  automatically  establishes  a  fixed 
connection) . 

The  general  rules  governing  substitutions  allow  a  great 
deal  of  flexibility  In  the  use  of  variable  connections.  For 
example,  If    a    and   (  b    are  both  variable  exits  and 

(b,^    ,    ('^oj         f        ^ '^'^'        ^^^   entrances,   (  ^  j   can 
also  be  connected  to  any  of  the  entrances  by  one  of  the 
substitutions 


•  b^  ,   ^  (  a  )  , 


or 


Furthermore,  If,  at  some  point  in  the  flow  diagram,  we  know 
that  (   b  ■   has  been  connected  to  one  of  the  entrances,  we  may 
wish  to  connect    a    to  that  same  entrance  without  committing 
ourselves  as  to  which  of  the  entrances  has  been  selected.  This 
is  done  by  a  substitution 


/- — ^ 
If  the  variable  symbol    b    has  previously  had  the  value 


-  20  - 


b^  )   ,    say,  assigned  to  it^,  the  above  substitution 


—  /" 

will  then  automatically  establish  a  connection  from   i  a 


to   (^2 

This  completes  the  discussion  of  general  flow  diagrams 
and  we  come  now  to  a  special  device  used  in  Monte  Carlo  work. 
In  Chapter  IV^  the  generation  of  random  variables  was  discussed, 
and  it  was  noted  that  curslvely  written  (script)  letters  are 
used  to  denote  probability  distributions j  in  particular,  the 
symbol  IX     denotes  the  uniform  distribution  of  a  single  variable 
in  the  interval   (O,  l).   The  expression 


\U}      ->  X 


means  that  a  number  in  the  Interval   (O,  l)   is  drawn  at  random 
from  M  (and  independently  of  all  such  numbers  previously  drawn 
from  it),  and  that  henceforth,  until  further  notice,  this  ntimber 
will  be  denoted  by  x.   This  symbolism  represents  the  idealized 
concept,  appropriate  for  an  abstract  description  of  a  Monte 
Carlo  calculation^  of  drawing  x  from  a  mathematically  random 
and  uniform  distribution?  in  practice,  it  is  approximated  by 
setting  X  equal  to  the  next  number  of  a  pseudo=random  sequence, 
or  in  some  similar  fashion. 

•  Similarly,  if   "n$"^"   (for  "spherical")  denotes  the 
distribution  of  unit  vectors  having  random  directions  in  three- 
dimensional  space,  the  expression 


SM     ->      T 
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means  that  a  vector  Is  dravm  from  this  distribution  and 
denoted  by   T  .   In  practice,  this  might  be  accomplished 
by  means  of  the  steps 


\/ 


i:uj     — >  y 


1 

>5 


>^2  ^  ^^2  ^x^^    <1 


No 


Yes 


}k_ 


where  T"-,,  'f'p,   r*    are  the  components  of  ?" ;   but  the  former 
expression  is  more  compact  and  allows  for  more  efficient  ways  of 
generating  the  distribution  "  ^f^" ,    If  they  can  be  found. 

When  such  conventions  are  used,  the  flow  diagram  describes 
an  abstract  stochastic  process,  or  Idealized  Monte  Carlo  procedure. 

There  Is  no  objection  to  the  use  of  vector  notation  In  flow 
diagrams;  the  rules  for  calculating  with  vectors  are  perfect- 
ly definite. 
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rather  than  a  deterministic  calculation.   In  use.  It  must  be 
"connected  to"  suitable  sources  of  random  variables,  which  In 
practice  are  not  quite  Ideal,  at  least  conceptually. 

In  the  margin  of  a  flow  diagram  (or  on  a  separate  sheet) 
one  should  Identify  all  symbols  denoting  special  probability 
distributions,  all  symbols  denoting  special  functions  (e.g. 
Bessel  functions),  etc.   The  flow  diagram  then  gives  a  concise 
specification  of  an  abstract  calculational  procedure.   Just  how 
well  this  procedure  can  be  carried  out  on  an  actual  computer 
may  still  depend  on  such  mundane  things  as  rounding  errors  and 
the  size  of  the  memory. 

3,  The  Estimation  of  Significance 

Until  it  has  been  ascertained  whether  rounding- errors 
have  destroyed  the  usefulness  of  a  calculation,  the  job  is  not 
finished.   The  task  of  a  computer  (electronic  or  human)  consists 
of  two  parts s  l)  performing  the  right  operations  on  the  right 
operands  in  the  right  orderj  2)  assessing  the  error  Introduced, 
at  each  stage,  by  the  imperfections  (limited  number  of  decimal 
or  binary  places)  s,  of  the  arithmetical  devices  used.   Most 
modern  machines  perform  part  l)  rapidly,  efficiently,  and  auto- 
matically and  do  nothing  whatever  about  part  2) .   In  this  section 
I  shall  discuss  various  methods  of  taking  care  of  part  2),  also 
automatically,  during  a  calculation. 

Note  1.   I  have  in  mind  problems  in  which  the  basic  mathe- 
matical context  is  that  of  the  real  number  field.   Work  with 
integers  can  of  course  be  done  exactly. 

-  23  - 


Note  2c  This  discussion  makes  sense  only  in  the  framework 
of  floating-point  work,  except  for  simple  problems  in  which  no 
scaling  is  required.   As  soon  as  numbers  have  to  be  shifted 
left  and  right  to  keep  them  in  scale,  automatic  procedures 
become  imperative,  if  one  wishes  to  know  what  is  really  happening 
to  significant  digits. 

Method  1 .   A  calculation  is  done  twice,  with  standard 
normalized  floating  point}  in  the  second  version,  the  treatment 
of  the  rightauost  bit  of  each  number  is  altered,  either  randomly 
or  by  some  procedure  different  from  the  standard  rounding. 
Comparison  of  the  two  sets  of  answers  gives  an  indication  of 
which  digits  are  significant.   This  method  is  advocated  by 
Bengt  Carlson  and  will  be  used  on  "STRETCH".  (See  also  Forsy the. '•'■') 

Method  2.   A  number  is  represented  by  a  set  of  three 
quantities;  a  fractional  part   M,   an  exponent   e,   and  a 
significance  index  s.   M  is  a  binary  or  decimal  fraction 
(^  <  |M|  <  1  or  j^  <  |M|  <  1,   respectively  -  i.e.,  the 
niimber  is  normalized) ,   s   is  an  integer   (>  O)   equal  to  the 
niunber  of  digits  of  M  that  are  significant  (or  alleged  to  be 
significant  according  to  a  certain  set  of  rules).   Together, 
M>   s,   and  e  represent  the  number   (M  +  2   )2   or 
{M  ±   10"^)10®. 

The  ryles  arei    1)    whenever  a  number  is  being  denormalized, 
in  preparation  for  addition  or  subtraction,  (this  happens  to 

o 

the  summand  with  the  smaller  exponent),  its  index  is  increased 

o 

but  never  outside  the  limits  (0,s  ),  where  s»  is  the  maximum 
value  of  s   allowed  by  the  word-  length  of  tne  machine. 
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by  the  same  amount  as  the  exponent,  so  that  the  Indicated  error 

2"^2®  or  10~^10®  Is  unchanged.   2)  then,  in  addition  or 

subtraction,  the  index  s  of  the  result  is  set  equal  to  the 

smaller  of  the  indices  of  the  s\immands  (after  step  1). 

J))   after  addition  or  subtraction,  during  normalizing  (in  case 

of  cancellation)  or  a  one-place  right  shift  (in  case  of  overflow), 

Q 

the  index  is  again  changed  by  the  same  simount  as  the  exponent. 

4)  In  multiplication  and  division,  after  the  complete  operation. 
Including  any  final  adjustment  for  overflow  or  normalizing^  the 
index  is  set  equal  to  that  of  the  less  accurate  operand. 

5)  Division  by  a  number  with  s  =  0  is  an  error,  regardless 
of  the  values  of  e  and  M,   for  then  zero  is  included  in  the 
possible  values  of  the  denominator  —  i.e.  in  the  range 

(M  +  1)2®   or  (M  +  1)10®. 

This  method  has  been  used  by  the  author ■■  •'  and  is  being 


[5} 


wired  into  the  "GEORGE"  machine  at  the  Argonne  National  Laboratory 

Method  J>.    Standard  floating  point  representation  is  used,  except 
that  each  number  is  allowed  to  have  Just  enough  leading  zeros  after 
the  binary  or  decimal  point  so  that  the  remaining  digits  are  all 
significant  (with  possible  exception  of  a  small  fixed  number  of 
guard  digits,  at  the  extreme  right,  to  absorb  the  accretion  of 
rounding  errors  -  of  course,  the  machine  need  not  know  which  digits 
are  guard  digits  and  which  are  truly  significant  ones.)   Addition 
and  subtraction  are  unnormallzedj  after  multiplication  and  division 
the  result  Is  shifted,  right  or  left,  until  it  has  exactly  the 
same  number  of  leading  zeros  as  the  less  accurate  operand,  and 
its  exponent  is  correspondingly  adjusted. 
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This  Method  has  been  used  by  Glenn  Lewis,'-  "■  and  a  very 
similar  method  Is  being  wired  Into  Manlao  III  at  Chicago. 

Method  k.    This  Is  the  physicist's  way  of  refining  method  2 
or  3>  it  is  reported  to  have  been  proposed  by  L.  H.  Thomas.   It 
is  probably  unnecessarily  refined  for  most  purposes  and  un- 
comfortably complicated  for  wiring  into  a  machine,  but  would 
occasionally  be  of  value  if  it  is  available  via  subroutines  or 
microprogramming.   A  number  is  represented  by  three  quantities 
M,   6M,   and  e,  where  M  and  6M  are  both  binary  or  decimal 
fractions}  the  number  represented  is   (M  +  6M)2®  or  (M  +  6M)10*, 
where  6M  denotes  the  probable  error.   When  numbers  are  combined 
arithmetically,  the  errors  of  the  operands  are  assumed  to  have 
normal  probability  distributions,  and  the  standard  formulas  are 
used  for  the  probable  error  of  the  result.   To  save  bits,   5M 
may  Itself  be  represented  as   (6m)2^   or   (5m)lO®  ,   where 
5m  Is  then  kept  with  an  accuracy  of  say  8-10  bits.   To  avoid 
root-extraction,  a  simple  approximation  is  used  for  vv^   +  y^  • 

Method  3'  ("Range  arithmetic")  This  is  the  mathematician's 
way  of  refining  method  2  or  3.   It  has  been  proposed  and  tested, 
by  use  of  subroutines,  by  (R?)  Moore. ''^''   A  number  X  Is 
represented  by  two  normalized  floating  point  quantities  7^, 
and  Xg*   which  detennlne  a  narrow  Interval  that  contains  the 
true  value  of  Xj  ^i  <  ^<  ^2*   (Normally,  the  exponents  of 
^-j^   and  Xg   would  be  equal  and  it  would  suffice  to  store  Just 
one  of  them.)  For  each  input  datum,  one  sets  X,  =  Xp  =  >• 
Irfhen  numbers  are  combined  arithmetically,  the  lower  limit  and 
the  upper  limit  of  the  result  are  calculated  separately  In  such 
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a  way  that  for  each  quantity  in  the  calculation  an  Interval  is 
obtained  in  which  its  true  value  is  guaranteed  to  lie. 

Method  60   Certain  specialized  problems  deal  with  rational 
numbers  only.   The  number  x^  ~   p/q  is  represented  by  the 
integers  p  and  q^,   which  are  stored.   In  addition  to  +!, 
-  g     X,  -^  s,      there  is  provision  for  reduction  to  lowest  terms 
(by  some  variation,  of  the  euclidean  algorithm).,   This  method 
may  possiblv  be  used  by  Charles  l4ytle  in  the  investigation  of 
certain  compatibility  conditions  connected  with  the  theory  of 
pseudo-analytic  functions.   I  learned  recently  of  an  application 
of  group  theory  to  quantum  mechanics ^  in  which  it  was  found 
useful  to  compute  with  a  similar  precise  representation  of 
certain  algebraic  numbers  in  a  machine o 

Comments o  a<,  Methods  2  and  5  appear  to  be  practically 
equivalent  in  performance..   They  have  both  been  tested.,  at 
NoYoUoD  on  problems  of  varying  complexity j,  using  power  series^ 
where  cancellation  Is  a  notorious  worry.   Judged  by  all  available 
checks  (including  problems  for  which  the  exact  solution  is  known), 
they  give  a  quite  accurate  indication  of  the  true  signif icancei 
the  number  of  actually  correct  digits  is  almost  always  the  same^ 
give  or  take  one^,  as  the  n-umber  claimed  to  be  significant. 
Similar  tests  have  been  reported  recently ^  for  method  2^  by 
workers  at  the  Argonne  National  Laboratory? '•-^■^  they  report  that 
the  method  is  equally  successful  for  problems  of  matrix  inversion. 

bo  In  a  problem  requiring  the  addition  of  a  very  large 
number  N  of  quantities  all  of  comparable  accuracy  and  magnitude, 
methods  2  and  3  would  underestimate  the  errors  by  falling  to  take 
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Into  account  their  accumulation,  which  tends  to  Increase  them 


by  about  v/N.   This  does  not  appear  to  happen  appreciably  In 
power-series  or  matrix  work,  and  I  suspect  that  It  only  rarely 
happens  otherwise,  but  it  might  happen,  for  example;,  in  the 
numerical  solution  of  an  ordinary  differential  equation  by 
very  fine  steps.   Then,  method  4  would  be  preferred., 

c.  For  work  in  the  physical  sciences,  where  errors  are 
estimated  (in  the  physicist's  sense)  rather  than  given  guaranteed 
bounds,  method  4  would  seem  to  be  preferable  to  method  5  if  an 
Improvement  over  method  2  or  3  is  really  needed y  because  the 
guaranteed  bounds  may  spread  apart,  during  a  calculation, 
considerably  more  rapidly  than  the  most  probable  bounds. 

do  Method  5  ought  to  be  available  for  occasional  rigorous 
tests  of  numerical  methods o   These  might  be  sufficiently  rare 
that  use  of  a  subroutine  for  the  range  arithmetic  would  suffice. 
However,  it  may  turn  out  that  it  is  precisely  in  very  long 
calculations  that  one  will  want  to  make  such  testsj  anyway,  the 
method  should  be  easily  achieved  in  a  micro-programming  system. 

e.  The  usefulness  of  method  6  (exact  representation  of 
rational  n\imbers)  is  likely  to  be  limited  not  only  by  the  presence 
of  irrational  operations  in  many  problems,  but  also  by  the  tendency 
of  the  numerators  and  denominators  to  become  progressively  larger 
during  a  calculation. 
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